A meaningful expansion around detailed balance 

Matteo Colangeli/ Christian Maes,^ and Bram Wynants^ 

' Dipartimento di Matematica, Politecnico di Torino, 
Cor so Duca degli Abruzzi 24, 10129 Torino, /to/J*] 
^Instituut voor Theoretische Fysica, K. U. Leuven, 3001 Leuven, Belgium 
^Institut de Physique Theorique, CEA-Saclay, 
F-91191 Gif-sur-Yvette Cedex, France 

Abstract 

We consider Markovian dynamics modeling open mesoscopic systems which are driven away from de- 
tailed balance by a nonconservative force. A systematic expansion is obtained of the stationary distribution 
around an equilibrium reference, in orders of the nonequilibrium forcing. The first order around equilib- 
rium has been known since the work of McLennan (1959), and involves the transient irreversible entropy 
flux. The expansion generalizes the McLennan formula to higher orders, complementing the entropy flux 
with the dynamical activity. The latter is more kinetic than thermodynamic and is a possible realization 
of Landauer's insight (1975) that, for nonequilibrium, the relative occupation of states also depends on the 
noise along possible escape routes. In that way nonlinear response around equilibrium can be meaningfully 
discussed in terms of two main quantities only, the entropy flux and the dynamical activity. The expansion 
makes mathematical sense as shown in the simplest cases from exponential ergodicity. 
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I. INTRODUCTION 



Recent years have seen an intensive search for a theoretical framework underlying nonequilib- 
rium fluctuations. The nature of nonequilibrium is diverse and rich but we hope to uncover some 
unifying structures that, ideally, would give an extension of the Gibbs formalism. Still, today, 
the mathematical models are very often simplified to Markov dynamics for jump or diffusion 
processes that represent open systems in weak contact with one or more reservoirs. These systems 
(such as molecular configurations) can be very small but nevertheless we have learnt how to attach 
thermodynamic meaning to various path-dependent quantities, see e.g. ll27l l33n. as they relate to 
what happens in the large environment. Most of that has concentrated around the concepts of 
energy and entropy, so that much of standard irreversible thermodynamics also got a formulation 
for mesoscopic systems. Another question has concerned response theory for these Markov 
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dynamics and some systematics have been obtained there also, e.g. in yj, |2|, 
A related issue concerns the characterization of the stationary distribution for a dynamics that 
breaks the condition of detailed balance. Of course we need a physically meaningful violation of 
detailed balance, and a useful interpretation is given by the condition of local (sometimes called. 



generalized) detailed balance, 15, 
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3411 . The latter requires that the power dissipated to 



the environment during a transition equals the logarithmic ratio between forward and backward 
transition rates. From there one can hope to find the corresponding stationary distribution in terms 
of these irreversible entropy fluxes. That is exactly what was achieved by McLennan in 1959, 
13011 . In that way, the stationary distribution picks up thermodynamic information and is no longer 
"just a stationary solution" of the Master equation. However, that McLennan proposal only works 
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, 2^. There does not appear to be a ready extension beyond the linear 
regime in terms of entropy considerations only. The present paper takes a next step but we need 
to go beyond the purely entropic concepts we are used to from heterogeneous equilibrium. That 
is, to go to second and higher order in an expansion around detailed balance we need another 
concept to complement the entropy fluxes. That novel quantity is called the dynamical activity 
and is much related to the notion of escape rate: it measures the reactivity and instability of a 
trajectory. Dynamical activity is thus much more concerned with kinetics than it is embedded into 
thermodynamics but by introducing it, we can complete the expansion beyond linear order in the 
nonconservative forces around equilibrium. In that sense, we add to the spirit of the McLennan 
proposal the insights of Landauer and others that for nonequilibria, the noise behavior along in- 
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and outgoing trajectories enters critically into the determination of state plausibilities, [Il5l-ll7ll. 



In the expansion of the stationary distribution, every term at any order in the nonequilib- 
rium forcing, just contains the same (dynamical) observables in various combinations of time- 
correlation functions under the reference equilibrium process. For example, in some precise sense 
correct up to third order we get 

p{x) = poix) [1 - (S): + ^{S Ti)l - ^(5 {S' + 37? - 12T2))1] 

for the stationary distribution p on states x in terms of the equilibrium distribution p^. The 
averages (■)° are over the detailed balance process started from state x while S is the irreversible 
entropy flux and 7i,72 denote the first and second order to the dynamical activity. These 
path observables S and 7i,2 depend of course on the nonequilibrium dynamics: S is basically 
determined by the work done by the nonconservative force in a particular trajectory, and 7i,2 
measures the expected dynamical activity long the followed trajectory. Specific details, rewriting 
and mathematical precision follow below. 



In the next section, we specify the Markovian set-up and we define the various objects such 
as p, S and T in the above. We concentrate on overdamped diffusions and jump processes for 
introducing (in a particular way) the nonequilibrium driving. Sections HlIl - llVl - IVl give the main idea 
and the structure of the expansion with some writing out for specific models to lowest order. The 
last section about nonlinear response suggests some immediate application before we conclude the 
paper. Appendix B defines the entropy flux and the dynamical activity for underdamped diffusions. 



II. SET-UP OF MARKOV STOCHASTIC DYNAMICS 



We restrict our analysis to Markovian stochastic models for mesoscopic systems driven by a 
nonconservative force, i.e., a force that cannot be derived from a potential. We imagine such 
systems to be immersed in an environment in thermal equilibrium at some inverse temperature 
(3. All conditions are time-independent and we want to characterize the statistical distribution of 
states for our open system when reaching stationarity. 

A first standard choice is to consider overdamped diffusions for state x G M", according to which 
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where the mobiUty x is a positive definite n x n-matrix not depending on x (for simpUcity only). 
The total force equals 

F = ef-VU 

for the energy U of the system, and with / the nonconservative force with amplitude e (our small 
number). When e = 0, the dynamics satisfies the condition of detailed balance, and given enough 
time the distribution of states converges to the equilibrium distribution Po{x) oc e~^^^^\ so we 
assume. 

Whenever e 7^ the system is not in equilibrium. The Fokker-Planck equation for the evolution 
of distributions jit is 

^(x) + V • [Xi^(x)/.,(x) - |V/.,(a:)] = 

The stationary solution p thus satisfies • F{x)p{x) — Ap(x), but we would be much helped 
by further more explicit, physical or systematic understanding of that p. One option is to find an 
expansion for that stationary distribution p{x) of the system in orders of e, assuming uniformly 
and exponentially fast relaxation behavior. That is the programme of the present paper and we 
will find that the main quantities in such an expansion are directly related to two specific path 
observables, that we now introduce. 

We fix a large time-interval [0, T] and we consider paths u — {xt, t e [0, T]), on which we define 

2 T-^ 

Tico) = dt/-x/-e/9^ dtf-x^U + eJ^ dtxV ■ f (2) 

Siuj) - e/3 r Axtof{xt) 
Jo 

The last stochastic integral (with the o) is in the sense of Stratonovich; in that way S is identified 
with /3 times the work done by the nonconservative force — for short, we speak about the entropy 
flux S. The quantity T is less familiar, and it contains both order e and order e^. Correspondingly, 
we write T — Ti +T2 for the first and second order. Its meaning is best understood before the 

continuum limit, in terms of jump processes to which we turn next. 

We consider a Markov jump process with discrete states x and as a reference we take jump rates 
ko{x, y) for the transition x ^ y which satisfy detailed balance, i.e., 

K{x, y) e-^^(-) = e-'^^(^) K{y, x) 

In this case the stationary Master equation is solved by po{x) oc exp(— {x)). To this reference 
we add an extra flux f{x, y) — —f{y, x) of energy in the transition x — > y, and write 

k,ix,y) = K{x,y)e^f^-'y^ (3) 
4 



To establish nonequilibrium we ask that the fluxes f{x, y) cannot aU be rewritten as the difference 
V{x) — V{y) of a unique potential V, which means that there are loops xi 0:2 a;„ = xi 

over which the sum f{xi,X2) + fix2,Xs) + . . . /(x„_i,x„) 7^ does not vanish. In terms of 
the condition of local detailed balance, these fluxes f{x,y) should be interpreted as the product 
of a displacement of a certain quantity and a nonconservative force, see [11], but here we do not 
need this formulation. The e is the magnitude of the nonequilibrium forcing. Finally, in ([3]) we 
have chosen to omit an extra symmetric prefactor ^peix, y) because the more general choice for ([3]) 
would be ke{x,y)/ko{x,y) =^pe{x,y) exp[/3e/(x, |/)/2]. We take however ?/) =il)e{y.,x) = 
1 for much greater simplicity. 

Again, for e 7^ 0, the stationary probability law p is only known indirectly as solution of the Master 
equation Yl,y[piy)^iy 1 ^) ~ p{x)k{x, y)] = for all x. 
This time the entropy flux is 

S{u) = ef3j2fi^t-,^t) (4) 

t<T 

as a sum over the jump times t in the trajectory w = (xs, s G [0, T]), and the dynamical activity is 

TM = 2 TdtJ^ A;o(x<,y)[e^^(^*'^) - 1] (5) 

Now we see better where the name activity comes from: T{co) is the difference in the escape rates, 
integrated over the trajectory. The escape rate k(x, y) measures the frequency by which the 
system exits state x, and in that way it counts the expected number of transitions away from x. 
In other words, T(a;) sees how the escape rate away from the trajectory u changes when adding 
the forcing /. In the appropriate rescaling the expression ^ is simply the continuum limit of dS]) 
from Markov jump to (overdamped) diffusions. We do that computation in Appendix A; the case 
of underdamped or inertial diffusions is shortly discussed in|Bl We repeat that we use here Q, 
i.e., with prefactor = 1; otherwise the expression for the activity gets more complicated — the 
major part of the analysis would however remain unchanged. 



ni. EXPANSION: MAIN IDEA 



The expansion of the stationary distribution starts from a simple idea which was applied al- 



ready in IlZ, 



121 



3,H 



the single-time distributions on states, and in particular the stationary 



distribution, can be obtained from its embedding in the path space distribution. The latter is the 
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distribution on the level of trajectories or paths u and gives the weight P{uj) for path-integrals. 
P is much more directly obtained and is much better-behaved than its projections on single time 
layers. In fact, we can give explicit expressions for the "action" A{uj) in 

P[uj) = e-^(") P°{uj) (6) 

that connects the distribution P on paths starting from po but with driving /, with the full equilib- 



rium reference distribution P°, see e.g. [|28n for some useful techniques. The action A is typically 



local in space-time and thus is similar to Hamiltonians or Lagrangians that we meet in (equilibrium 



statistical) mechanics, see e.g. |123i|27D. We can verify that 

A={r-S)/2 

as defined above for overdamped and jump processes. Furthermore, the action A in ([61) is left 
unchanged when both processes start from the same state x, 

P.M = e-^(-)p:(a;) (7) 

Now comes the embedding. As defined before, both nonequilibrium and equilibrium processes 
in ^ start at time t = from data distributed with the equilibrium po. At time T later, the 
probability to find the driven system in state x is 



p{x, T) = {S{xt — x)) 



PO 



averaging over the trajectories for the nonequilibrium dynamics. We assume that for T f +oo 
there is exponentially fast convergence p{x, T) — )■ p{x) (in the sense of densities) to the stationary 
distribution of the nonequilibrium process, uniformly in e. By ^ we can rewrite 

p{x, T) = {SixT - x) e-^("y (8) 

which is now an expectation value for the full equilibrium (detailed balance) process (and then we 
omit the subscript po). By time-reversal invariance, ([8]) equals 



pix,T) = (5(xo-vrx)e-^(^-)>° = p,(x) {e-^^'-%^^^ (9) 
where the time-reversal operator 9 acts as 

du = {{7lx)T-t,0 <t<T) 
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with -Kx equal to x except for flipping the velocities (if they are part of the state-description) or 
other variables with negative parity under time-reversal. In equilibrium po(7rx) = Po{x). Equality 
([9]) can still be rewritten in terms of S and T getting 



p{x,T)po{x) (e 



-(S+T)/2 



X0=7VX 



(10) 



Indeed, the decomposition A = (T — S)/2 follows the symmetry under time-reversal, 

S{uj) = A{euj) - A{u) 

r{u) = A{euj) + A{u) 

The fact that the quantity A(9uj) — A(uj) is the excess entropy flux from the system into the 
environment during the process u is one of the main discoveries for the construction of nonequi- 



2511 and the references in e.g. M, 
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librium statistical mechanics of the last decade, see y, 
Excess means the difference between the nonequilibrium process and the reference equilibrium 
process. Specifically, this excess is here equal to the work done by the nonequilibrium force ef. 



multiplied by (3. The dynamical activity T has been introduced and used before [|21 



2M but has 



no thermodynamic tradition. Its role is kinetic and that it influences the relative stability of states 
was somehow emphasized long before, cf. lilSMlTll . 



The left-hand side of (flOl) is assumed to converge exponentially fast to the stationary law p{x), 
but there is a problem with its right-hand side because both S and T are time-extensive of order 
T, being sent to infinity. We will therefore need to control the limit T f +oo and to worry about 
the exchange with the sum of the perturbation series. An important point here that follows from 
dV]), is the normalization 

{e(^-r)/2y^ = l (11) 

valid under the equilibrium process but started from an arbitrary state x. That itself can be ex- 
panded in orders of e, and takes care of many cancellations. Finally, one must use that S is 
anti- symmetric, and T is symmetric under time-reversal, so that over all time-intervals [0, T] 

(^-r'")° = 0, nodd (12) 

for all m and for all odd powers n. With these ingredients (fT0l)-(fTT1)-(fT2l). combined with fast 
relaxation for the equilibrium process, all is in place to start a systematic expansion. In the partic- 
ular cases we have in mind, see Section |nl the path function S is simply first order in e and T is 
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either T = T1 + T2 second order in e for diffusions, see ©, or of arbitrary order T = T1+T2 + ■■■ 
for Markov jump processes. For practical matters our expansion including second or third order 
around equilibrium is already new and relevant. 



IV. GENERAL EXPANSION 



Formally, expanding (fTOl) just gives 



On the other hand, (fTTI) gives 



2\ ' 1/2:0=772::? \ g 2 '^0 



Adding or subtracting these relations from the corresponding orders of the expansion of the sta- 
tionary distribution simplifies matters. In the end, in every order of the expansion we can choose 
that only those terms survive which are averages of quantities antisymmetric in time, 

""^"^'^^ ^ i-{sr„.+l{sr^r„ (13) 



Po[X) 



-^JSiS^ + 37?-l2%))l_ + 0{e^ 



The above considerations can be systematized. The formal expansion that results, after also taking 
into account the normalization in (fTT)) . is 



p(x,T) = po{x) 



00 / 

m=l ^ 



Xt^ = 'KX 



(14) 



where we introduced a shorthand notation for the path-dependent functionals Bm{G) acting on 
path observables G{uj^ e), defined via 

Bm{G) = Bm{Gi, Gm-k+l) 



h\...b^^k+i\ VTl) V(m-A; + 1)!,) ^^^^ 



k=l a- 

for Gi{uj) = d^G/de^{uj,0). The sum in (fT5l) extends over all sequences a of non-negative 
coefficients a = (61, bm-k+i), such that X]j=L'^^^ = ^ ^'^'^ Yl^=i^^ = Note that the 
Bm are versions of the so called complete Bell polynomials. There exist alternative, although 
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equivalent, expressions for (fT4l) . see e.g. On the other hand, the expression we use has the 
advantage of being compact and suitable for numerical implementation. 

From (fT4l) and (fT5l) , we can write the explicit expression for the m-th order in e. Remember 
that we write T = 71 + 72 + • • • and that S is of order e while 7^ is of order e". The result is 

^ ' m=l am 

The sum in (fT6l ) extends over all sequences dm of non-negative integers (6o, &i, bm), such that 
6o is odd and bo + J2f=i Jbj = fn. For illustration to construct (fT3] ). m = 1 requires &o = 1 and 
all other bj = 0; m = 2 requires bo = bi = 1 with all the other bj = 0; m = 3 allows three cases 
bo = b2 = 1, bo = l,bi = 2 and 6o = 3 each time with all other bj = 0. 

In the case of diffusions (where p{x, T) must be understood as a probability density with respect 
to dx), see Q, we have 7^j = forn > 2 so that we must then also require bj = 0, j > 3 in each 

The first important thing to observe about the expansion (fT6l) is that it converges for fixed state 
X and uniformly in time T f oo if there is c = c(x) < oo such that each term is bounded like 

\ / XO=TTX 

The reason is that 

^ ^^2' bo\bi\...bJ. - ^ ^ 2^k\ 
converges for small enough e. 

We only argue for (flTI) explicitly for the first and the second order. The first order is the McLennan 
formula (see immediately below) and has been treated before in [2^. The second order adds a new 
complication (to be treated below) and that complication is repeated for the higher order terms and 
can be solved in the same way. At any rate we cannot quite leave it with (fT3l) or with (fT6l) because 
we are interested in the limit T f oo and 5, T do not make any sense in that limit. We thus need a 
further rewriting for which we need some more model-dependent input and to which we turn next. 
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V. EXPANSION DETAILS 



A. McLennan formula: the first order 



The first order in the expansion of the stationary distribution (fT3l) has been known for a long 



time [|30l1 and has been reconsidered more recently in [|12l.l26h. In fact, the idea of obtaining the 
McLennan-formula via the embedding described under Section |lll] originates from lll2ll . The way 
how to deal with the limiting behavior T f oo, e | was treated in 1I26I1 . We briefly repeat this and 
we concentrate on the Markov processes of Section|Ill 



The main point is that 



{S{uj)): = eP / dt{w{xt)): 



(18) 



where, for jump processes w(x) = J2yf^oix,y)f{x,y), and for overdamped diffusions 
w = xV ■ f / /3 — xf ' Vt/, see [|26ll . The expression (fTSl) allows the limit T ^ 00 uniformly in e 
once we assume the equilibrium process to be irreducible and exponentially ergodic. 



Plugging (fT8l) into (fT6l) thus gives the linear order expression 



p{x)/po{x) = l-e(3h{x) + 0{e^) 



(19) 



with 



h{x) 



dt {w{x,)r. 



in which the integral is exponentially convergent. The relation with local equilibrium distributions 
is also discussed in ll^l . 



B. Second order 



We look at the m = 2 term in (fT6l) . The main object to consider for jump processes is 

/ ( V/(a;t-,a;i) y'fco(a;.,l/)/(a;s,2/) 

where the sum is over the jump times. On the other hand, for overdamped diffusions we can 
introduce a{xt,dxt) = e/Sdxt o f{xt) and Ti{x) = —[5ef{x) ■ xVf/ + e^V ■ /, so that we must 
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deal then with 

rT 



h2{x) 



[ ds [ {a{xt,dxt)ri{xs))l (20) 
Jo Jo 

We can keep with this "diffusion-"notation also for the "jump-"case if we then take 
a{xt,dxt) = e(3 dJ\ftf{xt- , Xt) where J\ft is the Poisson process counting jumps, and Ti{x) = 
e/3 ko{x, y)f{x, y). We give the argument for the jump-case. 

Because we are dealing with a correlation function of two time-extensive quantities, we have 
to use twice the exponential convergence to the equilibrium expectation. We will use the bound 
\{fi^u)9ixo)y — {f{xu))° {g{xQ))°\ < CfCgC'"^, for some positive a and where Cj and Cg 
bound the functions /, respectively g. 

To this end, we split the s-integral in (l20l) over [0, t] U [t, T]. For the first part, 

ds {a{xt,dxt)Ti{xs))l= dt ds {Tiixt)Ti{xs))l 

Jo Jo 



JO 



as follows from (fTTI) . For the second part we use 







' ds {(r{xt,dxt)Ti{xs))l / dt / {a{xs,dXs)Ti{xt))l 
t Jo Jo 



f dt [\[a{x,,dxs) + ri{xs)ds]ri{xt))l (21) 
Jo Jo 



so that always s < t in what follows. Therefore, 

h2{x) 

First of all, the integrand has an equilibrium expectation equal to zero, as one can see from time- 
reversal invariance of equilibrium: 

{[a{xs,dxs) + Ti{xs)ds]Ti{xt)y = {[-a{xt,dxt) + Ti{xt) dt]Ti{xs))° 

As t is always bigger than s in the integrand of (|2T1) . we can do the same trick as before by replacing 
{(j{xt, dxt)Ti{xs)Y by {Ti{xt)dtTi{xs))° . This shows us indeed that the equilibrium expectation 
of the integrand in (|2T]) is zero. We can make that more explicit by substituting the expressions for 
the jump-case, to have 

{[ij{xsAxs) + Ti{xs)ds\Ti{xt))l = 

{ef3Y{[dAfJ{xs^,x,) + axs)ds]{axt)\x,)X (22) 

for ({x) = J2y ko{x, y)f{x, y). Therefore, with Ps{x, y) the transition probability for the detailed 
balance reference dynamics to find y at time s when starting from x, 

{[a{x,Axs) + ri{xs)ds\ri{xt))l = (23) 
{e(3f ds ^[ps{x, z) ko{z, y) f{z, y) + y) ko{y, z)f{y, z)](C(xt_,))° 
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which indeed vanishes upon replacing Ps{x, y) — )■ Poiy) by the equilibrium distribution because 
Po{z) ko{z, y) f{z, y) + poiy) Kiy, z)f{y, z) = by detailed balance. 
Furthermore, the equilibrium expectation of 71 is zero, (Ji{xt)Y = 0, so that 

|(C(a;,_.))°| <C(i/)e-"(*-^) 

The rest of the argument is straightforward by using that Ps{x, y) is exponentially close to Po{y) 
as a function of the time s: 

\h2{x)\< [ [ ds e"°(*-')e-"''C"(x) 
Jo Jo 

which is clearly finite as T f cxo. 



VI, APPLICATION TO NONLINEAR RESPONSE 

One of the very first applications of the McLennan formula (the first order as in (fT9l) ) is the 
derivation of linear response around equilibrium. It is indeed possible to derive the Green-Kubo 



relations from it; see e.g. Section IIIB in n26\ . The analogue can be done also for higher order 
response as we now indicate. 

We imagine the system in equilibrium up to time zero. At that time an external stimulus ef is 
added, as modeled in the set-up of Section UIl driving the system away from equilibrium. We can 
then estimate, using (fT6l) say to second order in e, for an observation Q at time T, 

(QM)^o = {Qr-{Q{7rxo)S{io)r+^{Q{nxo)S{u)ri{u)r + o{e') 
= {Qr + {Q{xT)S{uj)r - ^{Q{xT)S{uj)T,{uj)y + o{e') 

When the perturbation ef is the difference or the gradient of a potential eV, then the entropy flux 
5* = /3e[F(sT) — V"(a;o)]5 assuming that = V^irx) (see formulae Q and (H))). The expectation 
value then reads, up to second order, 

(QM)^o = {Qr + f3e{Q{xT)[V{xT) -V{xo)]r 

/ ds{Q{xT)[V{xT)-V{xo)]LoV{xs)r (24) 
^ Jo 

The first order term is consistent with the (equilibrium) fluctuation-dissipation theorem. In the 
second order term in (|24)) we have used the expression for the backward generator LoV{x) = 
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T,ykoix,y)[V{y) - V{x)] for jump processes, LoV{x) ■ 
processes, to rewrite 

ri(u;) = e/3 [ dsLoV{x 



-XVU ■ VV + x^V/f3 for diffusion 



when f{x, y) = V{y) — V{x) (in the jump-case for ipe{x, = 1 in ([3])) or when f{x) 
(in the diffusion-case Q). As a consequence. 



W{x) 



PO 



e=0 



T 



I ds{[Q{xT) - Q{nxo)][VixT) -Vixo)]LoVixs)y 



Observe that the terms in the expansion (l24l) are not just expectations of observables at one fixed 
time; they are correlation functions. It is therefore natural to investigate also the perturbation 
expansion of correlation functions in the nonequilibrium process. That follows most generally 
from expanding the exponential and the action in ©-([T]). It is then straightforward to check e.g. 
that 

(25) 



de 



PO 



e=0 



P g-^{{[Q{Xt) - Q{nXo)][V{Xt) - r(Xo)]) 



PO 



e=0 



The treatment of nonlinear response can be started differently, for example from applying Ward- 
Takahashi type identities starting from the fluctuation symmetry in the distribution of the entropy 
flux, see e.g. Section 10 in [23] or more recently in yj. 
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291, 



311, 



3511 . The advantage of the 



present treatment however is that all the terms in the expansion are explicitly expressed as cor- 
relation functions in the full equilibrium reference process. On the other hand, we do not know 
how to extend our ideas to deterministic dissipative dynamics. There the steady state attractor has 
a lower dimension than the embedding space and the invariant density becomes singular. That 
aspect probably becomes less stringent for macroscopic systems, or, depending on the reduced de- 
scription and the nature of the observables one can attempt a suitable projection on some smooth 
manifold. For a perturbation theory featuring a first order term producing the standard fluctuation- 
dissipation theorem and higher order terms giving rise to nonlinear response, see e.g. [|i andU 
for two different approaches. For further convergence of ideas, it would be very helpful to iden- 
tify the notion of dynamical activity within the thermodynamic formalism of smooth dynamical 
systems. One natural guess would proceed via the escape rate formalism as in 



VII. CONCLUSION 



Expansions for nonequilibrium mesoscopic systems can be made in various ways. If one 
is interested in the stationary distribution away from detailed balance, as we are, we can try 
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Bom-type or Dyson-type perturbation expansions starting from the Master equation. These can 
certainly be computationally useful, but that however is not the main point of this present paper. 
What was attempted here is to give an expansion in terms of few (basically two) path-dependent 
physical observables, the entropy flux and the dynamical activity. It is in the same spirit of many 
attempts dating already from the 1950-60's where a major issue was to understand whether the 
stationary nonec 



only, 118 



milibrium distribution could be described in terms of macroscopic parameters 
30n . Here we deal with mesoscopic systems and we incorporate more recent ideas 
concerning the importance of the kinetics (noise and activity) for finding the relative probability 
of states, iIITII . The utility of the expansion depends on the relevance of nonlinear response around 
equilibrium for stochastic Markovian dynamics, and whether natural phenomena obey the various 
conditions imposed. 



The starting point of our analysis is a path-integral formulation for the distribution of histories 
in terms of a reference equilibrium distribution. The breaking of detailed balance introduces irre- 
versible fluxes, making up the time anti-symmetric term in the action, and also creates excesses in 
dynamical activity, governing the time-symmetric fluctuations. Both fluxes and activities can be 
written down explicitly as function of the system's trajectory for Markov jump and diffusion pro- 
cesses, where the physical interpretation is supported by the condition of local detailed balance. 
The subsequent expansion then mimics a high-temperature expansion for Gibbs distributions but 
now on space-time, where the exponential function gets Taylor-expanded. Additional symme- 
tries simplify the expansion and fast relaxation properties enable full control over the terms in the 
expansion, as explicitly demonstrated for the first few. It remains however an interesting mathe- 
matical challenge to rigorously prove the convergence of the complete perturbation series under 
an appropriate minimal set of conditions on the dynamics. 
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Appendix A: Continuum limit of dynamical activity 



There is a large literature concerning the derivation of Langevin dynamics or the Fokker- Planck 
equation from the Master equation for jump processes. Most of the time one develops around 
some small parameter like the inverse volume or the inverse number of particles in what is known 
as the Kramers-Moyal expansion, We do not repeat that here, but since the dynamical 

activity is still relatively unknown we show how to obtain Q from Q in a continuum limit. 

Consider a small mesh 5 > and a walker on a ring with detailed balance transition rates 

koix,x±6) = D ex-p-^[U{x±5) - U{x)\ D>0 

and with driven rates 

k{x, X ± 6) = ko{x, X ± 6) exp ^-^f{x, x ± 6) 
The excess in dynamical activity when in state x is then, following ([5]), 

k{x, X + S) — ko{x, X + S) + k{x, X — S) — ko{x, x — S) 

which we expand to order 5^ as 

Dil- ^U\x)) /(x, x + 5) + ^^fix, X + S)] 
+ Dil + ^U'{x)) /(x, x-5) + ^^fix, X - S)] 
= ^^^^ [fix, x + 6) + fix, x-6)] + ^ [fix, x + S)-fix- S, x)] 

o Z 

^ — U'ix) [fix, X + 6) + fix - 5, x)] 

In other words, setting also D(3 = x and fix,x) = fix), 
2 

lim — [k{x, X + 6) — koix, x + 6) + k{x, x — 6) — koix, x — 6)] = 
= ^e'fix) + xef'ix)-xf3efix)U'ix) (Al) 

in which we recognize the instantaneous excess in dynamical activity Q. 
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Appendix B: Underdamped diffusion processes 



We consider here a Langevin dynamics for a particle with mass m, position qt and velocity Vt: 

dqt = Vtdt 

mdvt = [F{qt) - m'yvtldt + V2DdBt (Bl) 

7 is the friction coefficient and Bt is a standard Wiener process giving rise to Gaussian white noise. 
The (symmetric) matrix D governs the variance of that noise. The Einstein relation between 7 and 
D brings in the inverse temperature: 7 = f3D. The corresponding Fokker-Planck equation for the 
evolution of distributions fit is 



d_ 
dt 

for V = (V,, V„) and for current = {J^, JJ[) with 



/it + V ■ J^, = (B2) 



= mvfi, = Ffi — 7 mv jj, — D (B3) 

to be understood with diagonal matrices 7 and D. As before we write the force F = ef — V qU . 
The equilibrium case e = has the equilibrium distribution 



Here the action is 



A[uj) = -log 



dV%uj) 
r{cu) - S{co) 



with 



2 pT pi pi 

r{u) = ^ dtf-D~^f-e / dtf ■ D-^VU -me / dvtoD'^f 
2 Jo Jo Jo 



(B4) 



T 



S{uj) = e/3 / dtvf (B5) 







One checks that S equals the work done by the nonconservative force /, times /3. The dynamical 
activity T consists of several terms with less obvious physical meanings. Still, it consists of in 
principle measurable quantities: forces on the one hand and D on the other hand, which depends 
on the friction coefficient and the mass of the particle. This dynamical activity turns up in the 
linear response around nonequilibrium, see tM- A natural continuation of Appendix A would be 
to understand the Smoluchowski limit of (IB4I) . We don't do that here but the expansion of Section 
|IV]and the formula (fT6l) remain unchanged. 
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